System and method for determining a uniform external magnetic field

ABSTRACT

A processing apparatus, optionally integrated into a device having a plurality of sensors including a magnetometer, generates navigational state estimates for the device. The processing apparatus has a magnetometer-assisted mode of operation in which measurements from the magnetometer are used to estimate the navigational state and an alternate mode of operation in which the navigational state of the device is estimated without measurements from the magnetometer. For a respective time period, the processing apparatus operates in the alternate mode of operation. During the respective time period, the processing apparatus collects a plurality of magnetometer measurements and determines whether they meet measurement-consistency requirements. If the measurements meet the measurement-consistency requirements, the processing apparatus transitions to the magnetometer-assisted mode of operation. If the measurements do not meet the measurement-consistency requirements, the processing apparatus continues to operate in the alternate mode of operation.

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application Ser.No. 61/615,327, filed Mar. 25, 2012, which application is incorporatedby reference herein in its entirety.

TECHNICAL FIELD

The disclosed embodiments relate generally to determining a navigationalstate of a navigation sensing device.

BACKGROUND

A navigation sensing device detects changes in navigational state of thedevice using one or more sensors. In some situations sensor measurementsfrom multiple sensors are combined to determine a navigational state ofthe sensing device. The navigational state of the device can be used formany different purposes, including controlling a user interface (e.g.,moving a mouse cursor) and tracking movements of the navigation sensingdevice over time.

SUMMARY

However, in situations where sensor measurements from multiple sensorsare combined to determine the navigational state (e.g., attitude andposition) of the navigation sensing device, the inaccuracy of one of thesensors can distort the navigational state even when other sensors areaccurate. For example a non-uniform disturbance in a local magneticfield (e.g., caused by electrical wiring or proximity to a largemetallic object) will degrade the performance of a magnetometer but willnot affect an accelerometer or a gyroscope. Thus, ceasing to use sensormeasurements from the magnetometer to determine the navigational stateof the device when a magnetic disturbance is detected will improve thenavigational state estimate by reducing or eliminating the effect of themagnetic disturbance on the navigational state estimate. However, anaccelerometer and/or gyroscope measurements are prone to drift (e.g.,accumulated error in the estimate of the navigational state). Usingsensor measurements from a magnetometer helps prevent navigational statedrift in situations where the reference magnetic field (e.g., theEarth's magnetic field) is locally constant.

Thus, in many situations is advantageous to resume using sensormeasurements from the magnetometer to estimate the navigational state ofthe device after the local magnetic disturbance has ended (e.g., becausethe disturbance has ceased or because the device has been moved out ofrange of the disturbance). However, reintroducing the use ofmagnetometer measurements while the uncompensated magnetic disturbanceis still present will distort the navigational state estimate.Accordingly, it would be desirable to be able to determine whether theuncompensated magnetic disturbance is still present. However variousproblems can complicate the determination as to whether theuncompensated magnetic disturbance is still present, including thepossibility that while the magnetometer was not in use, the navigationalstate estimate drifted so that the device continues to register anuncompensated magnetic disturbance even after the uncompensated magneticdisturbance has ceased. As such, it would be advantageous to cease usingmeasurements from a magnetometer to generate navigational stateestimates in the presence of an uncompensated magnetic disturbance andintelligently determine whether or not to resume using measurements froma magnetometer to generate navigational state estimates for a device.

Some embodiments provide a method for, at a processing apparatus havingone or more processors and memory storing one or more programs that whenexecuted by the one or more processors cause the respective processingapparatus to perform the method. The method includes generatingnavigational state estimates for a device having a plurality of sensors.The plurality of sensors including a magnetometer. The processingapparatus has a plurality of modes of operation including: amagnetometer-assisted mode of operation in which measurements from themagnetometer are used to estimate the navigational state of the deviceand an alternate mode of operation in which measurements from a subsetof sensors of the plurality of sensors that does not include themagnetometer are used to estimate the navigational state of the device.The method further includes, for a respective time period, operating inthe alternate mode of operation and during the respective time periodcollecting a plurality of magnetometer measurements and determiningwhether the plurality of magnetometer measurements meet predefinedmeasurement-consistency requirements. The method also includes, inaccordance with a determination that the plurality of magnetometermeasurements meet predefined measurement-consistency requirements,transitioning to the magnetometer-assisted mode of operation and inaccordance with a determination that the plurality of magnetometermeasurements do not meet predefined measurement-consistencyrequirements, continuing to operate in the alternate mode of operation.

In accordance with some embodiments, a computer system (e.g., anavigation sensing device or a host computer system) includes one ormore processors, memory, and one or more programs; the one or moreprograms are stored in the memory and configured to be executed by theone or more processors and the one or more programs include instructionsfor performing the operations of any of the methods described above. Inaccordance with some embodiments, a non-transitory computer readablestorage medium (e.g., for use by a navigation sensing device or a hostcomputer system) has stored therein instructions which when executed byone or more processors, cause a computer system (e.g., a navigationsensing device or a host computer system) to perform the operations ofany of the methods described above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system for using a navigation sensing device,according to some embodiments.

FIG. 2 is a block diagram illustrating an example navigation sensingdevice, according to some embodiments.

FIGS. 3A-3E are block diagrams illustrating configurations of variouscomponents of the system including a navigation sensing device,according to some embodiments.

FIG. 4 is a diagram illustrating an example of switching between amagnetometer-assisted mode of operation and an alternate mode ofoperation, according to some embodiments.

FIGS. 5A-5D are flow diagrams of a method for determining estimatednavigational states of a navigation sensing device, according to someembodiments.

FIG. 6 presents a block diagram of an example navigation sensing device,according to some embodiments.

FIG. 7 presents a block diagram of an example host computer system,according to some embodiments.

Like reference numerals refer to corresponding parts throughout thedrawings.

DESCRIPTION OF EMBODIMENTS Exemplary Use Cases

Navigation sensing devices (e.g., human interface devices or motiontracking device) that have a determinable multi-dimensional navigationalstate (e.g., one or more dimensions of displacement and/or one or moredimensions of rotation or attitude) are becoming increasingly common forproviding input for many different applications. For example, such anavigation sensing device may be used as a multi-dimensional pointer tocontrol a pointer (e.g., a cursor) on a display of a personal computer,television, gaming system, etc. As another example, such a navigationsensing device may be used to provide augmented reality views (e.g., byoverlaying computer generated elements over a display of a view of thereal world) that change in accordance with the navigational state of thenavigation sensing device so as to match up with a view of the realworld that is detected on a camera attached to the navigation sensingdevice. As yet another example, such a navigation sensing device may beused to provide views of a virtual world (e.g., views of portions of avideo game, computer generated simulation, etc.) that change inaccordance with the navigational state of the navigation sensing deviceso as to match up with a virtual viewpoint of the user based on theorientation of the device. In other situations a navigation sensingdevice may be used as a motion tracking device to track changes inposition and/or orientation of the device over time. These trackedchanges can be used to map movements and/or provide other navigationalstate dependent services (e.g., location or orientation based alerts,etc.). In this document, the terms orientation, attitude and rotationare used interchangeably to refer to the orientation of a device orobject with respect to a frame of reference.

In order to function properly (e.g., return results to the user thatcorrespond to movements of the navigation sensing device in predictableways), these applications rely on sensors that determine accurateestimates of the navigational state of the device. While specific usecases are described above and will be used to illustrate the generalconcepts described herein, it should be understood that these examplesare non-limiting examples and that the embodiments described hereinwould apply in an analogous manner to any navigation sensing device thatwould benefit from an accurate estimate of the navigational state of thedevice.

System Overview

Attention is now directed to FIG. 1, which illustrates an example system100 for using a navigation sensing device (e.g., a human interfacedevice such as a multi-dimensional pointer) to manipulate a userinterface. As shown in FIG. 1, an example Navigation Sensing Device 102(hereinafter “Device 102”) is coupled to a Host Computer System 101(hereinafter “Host 101”) through a wireless interface, according to someembodiments. In these embodiments, a User 103 moves Device 102. Thesemovements are detected by sensors in Device 102, as described in greaterdetail below with reference to FIG. 2. Device 102, or Host 101,generates a navigational state of Device 102 based on sensormeasurements from the sensors and transmits the navigational state toHost 101. Alternatively, Device 102 generates sensor measurements andtransmits the sensor measurements to Host 101, for use in estimating anavigational state of Device 102. Host 101 generates current userinterface data based on the navigational state of Device 102 andtransmits the current user interface data to Display 104 (e.g., adisplay or a projector), which generates display data that is displayedto the user as the currently displayed User Interface 105. While Device102, Host 101 and Display 104 are shown in FIG. 1 as being separate, insome embodiments the functions of one or more of these elements arecombined or rearranged, as described in greater detail below withreference to FIGS. 3A-3E.

Thus, the user can use Device 102 to issue commands for modifying theuser interface, control objects in the user interface, and/or positionobjects in the user interface by moving Device 102 so as to change itsnavigational state. In some embodiments, Device 102 is sensitive to sixdegrees of freedom: displacement along the x-axis, displacement alongthe y-axis, displacement along the z-axis, yaw, pitch, and roll.

In some other situations, Device 102 is a navigational state trackingdevice (e.g., a motion tracking device) that tracks changes in thenavigational state of Device 102 over time but does not use thesechanges to directly update a user interface that is displayed to theuser. For example, the updates in the navigational state can be recordedfor later use by the user or transmitted to another user or can be usedto track movement of the device and provide feedback to the userconcerning their movement (e.g., directions to a particular locationnear the user based on an estimated location of the user). When used totrack movements of a user without relying on external locationinformation (e.g., Global Positioning System signals), such motiontracking devices are also sometimes referred to as pedestrian deadreckoning devices.

In some embodiments, the wireless interface is selected from the groupconsisting of: a Wi-Fi interface, a Bluetooth interface, an infraredinterface, an audio interface, a visible light interface, a radiofrequency (RF) interface, and any combination of the aforementionedwireless interfaces. In some embodiments, the wireless interface is aunidirectional wireless interface from Device 102 to Host 101. In someembodiments, the wireless interface is a bidirectional wirelessinterface. In some embodiments, bidirectional communication is used toperform handshaking and pairing operations. In some embodiments, a wiredinterface is used instead of or in addition to a wireless interface. Aswith the wireless interface, the wired interface may be a unidirectionalor bidirectional wired interface.

In some embodiments, data corresponding to a navigational state ofDevice 102 (e.g., raw measurements, calculated attitude, correctionfactors, position information, etc.) is transmitted from Device 102 andreceived and processed on Host 101 (e.g., by a host side device driver).Host 101 uses this data to generate current user interface data (e.g.,specifying a position of a cursor and/or other objects in a userinterface) or tracking information.

Attention is now directed to FIG. 2, which illustrates an example ofDevice 102, according to some embodiments. In accordance with someembodiments, Device 102 includes one or more Sensors 220 which producecorresponding sensor outputs, which can be used to determine anavigational state of Device 102. For example, in one implementation,Sensor 220-1 is a multi-dimensional magnetometer generatingmulti-dimensional magnetometer measurements (e.g., a rotationmeasurement), Sensor 220-2 is a multi-dimensional accelerometergenerating multi-dimensional accelerometer measurements (e.g., arotation and translation measurement), and Sensor 220-3 is a gyroscopegenerating measurements (e.g., either a rotational vector measurement orrotational rate vector measurement) corresponding to changes inorientation of the device. In some implementations Sensors 220 includeone or more of gyroscopes, beacon sensors, inertial measurement units,temperature sensors, barometers, proximity sensors, single-dimensionalaccelerometers and multi-dimensional accelerometers instead of or inaddition to the multi-dimensional magnetometer and multi-dimensionalaccelerometer and gyroscope described above.

In some embodiments, Device 102 also includes one or more of: Buttons207, Power Supply/Battery 208, Camera (not shown in the example in FIG.2; see “camera(s)” 1180 in FIG. 6) and/or Display 216 (e.g., a displayor projector). In some embodiments, Device 102 also includes one or moreof the following additional user interface components: one or moreprocessors, memory, a keypad, one or more thumb wheels, one or morelight-emitting diodes (LEDs), an audio speaker, an audio microphone, aliquid crystal display (LCD), etc. In some embodiments, the variouscomponents of Device 102 (e.g., Sensors 220, Buttons 207, Power Supply208, Camera (not shown in FIG. 2), and Display 216) are all enclosed inHousing 209 of Device 102. However, in implementations where Device 102is a pedestrian dead reckoning device, many of these features are notnecessary, and Device 102 can use Sensors 220 to generate trackinginformation corresponding changes in navigational state of Device 102and transmit the tracking information to Host 101 wirelessly or storethe tracking information for later transmission (e.g., via a wired orwireless data connection) to Host 101.

In some embodiments, one or more processors (e.g., 1102, FIG. 6) ofDevice 102 perform one or more of the following operations: samplingSensor Measurements 222, at a respective sampling rate, produced bySensors 220; processing sampled data to determine displacement;transmitting displacement information to Host 101; monitoring thebattery voltage and alerting Host 101 when the charge of Battery 208 islow; monitoring other user input devices (e.g., keypads, buttons, etc.),if any, on Device 102 and, as appropriate, transmitting informationidentifying user input device events (e.g., button presses) to Host 101;continuously or periodically running background processes to maintain orupdate calibration of Sensors 220; providing feedback to the user asneeded on the remote (e.g., via LEDs, etc.); and recognizing gesturesperformed by user movement of Device 102.

Attention is now directed to FIGS. 3A-3E, which illustrateconfigurations of various components of the system for generatingnavigational state estimates for a navigation sensing device. In someembodiments, there are three fundamental components to the system fordetermining a navigational state of a navigation sensing devicedescribed herein: Sensors 220, which provide sensor measurements thatare used to determine a navigational state of Device 102, MeasurementProcessing Module 322 (e.g., a processing apparatus including one ormore processors and memory) which uses the sensor measurements generatedby one or more of Sensors 220 to generate estimates of the navigationalstate of Device 102 which can be used to determine current userinterface data and/or track movement of Device 102 over time, and,optionally, Display 104, which displays the currently displayed userinterface to the user of Device 102 and/or information corresponding tomovement of Device 102 over time. It should be understood that thesecomponents can be distributed among any number of different devices.

In some embodiments, Measurement Processing Module 322 (e.g., aprocessing apparatus including one or more processors and memory) is acomponent of the device including Sensors 220. In some embodiments,Measurement Processing Module 322 (e.g., a processing apparatusincluding one or more processors and memory) is a component of acomputer system that is distinct from the device including Sensors 220.In some embodiments a first portion of the functions of MeasurementProcessing Module 322 are performed by a first device (e.g., raw sensordata is converted into processed sensor data at Device 102) and a secondportion of the functions of Measurement Processing Module 322 areperformed by a second device (e.g., processed sensor data is used togenerate a navigational state estimate for Device 102 at Host 101).

As one example, in FIG. 3A, Sensors 220, Measurement Processing Module322 and Display 104 are distributed between three different devices(e.g., a navigation sensing device such as a multi-dimensional pointer,a set top box, and a television, respectively; or a motion trackingdevice, a backend motion processing server and a motion trackingclient). As another example, in FIG. 3B, Sensors 220 are included in afirst device (e.g., a multi-dimensional pointer or a pedestrian deadreckoning device), while the Measurement Processing Module 322 andDisplay 104 are included in a second device (e.g., a host with anintegrated display). As another example, in FIG. 3C, Sensors 220 andMeasurement Processing Module 322 are included in a first device, whileDisplay 104 is included in a second device (e.g., a “smart”multi-dimensional pointer and a television respectively; or a motiontracking device and a display for displaying information correspondingto changes in the movement of the motion tracking device over time,respectively).

As yet another example, in FIG. 3D, Sensors 220, Measurement ProcessingModule 322 and Display 104 are included in a single device (e.g., amobile computing device, such as a smart phone, personal digitalassistant, tablet computer, etc., with an augmented realityapplication). As a final example, in FIG. 3E, Sensors 220 and Display104 are included in a first device (e.g., a game controller with adisplay/projector), while Measurement Processing Module 322 is includedin a second device (e.g., a game console/server). It should beunderstood that in the example shown in FIG. 3E, the first device willtypically be a portable device (e.g., a smartphone or a pointing device)with limited processing power, while the second device is a device(e.g., a host computer system) with the capability to perform morecomplex processing operations, or to perform processing operations atgreater speed, and thus the computationally intensive calculations areoffloaded from the portable device to a host device with greaterprocessing power. While a plurality of common examples have beendescribed above, it should be understood that the embodiments describedherein are not limited to the examples described above, and otherdistributions of the various components could be made without departingfrom the scope of the described embodiments.

Using Multiple Sensors to Estimate Navigational States

In some implementations measurements from multiple sensors are used toestimate navigational states of Device 102 (e.g., via sensor fusion).For example, one combination of sensors that provide measurements thatcan be used to estimate navigational state (e.g., orientation and/orposition) includes a gyroscope, one or more accelerometers, and one ormore magnetometers. This navigational state data is used by otherprocesses, such as pedestrian dead reckoning which uses changes in thenavigational state over time to determine movement of Device 102.

Sometimes, sensor measurements from a respective sensor cannot betrusted because the sensor measurements differs too much from theexpected model of sensor behavior for the respective sensor. Forexample, in many situations it is difficult or impossible to modeltranslational acceleration for an accelerometer, and under the conditionthat there is translational acceleration present, sensor measurementsfrom the accelerometer cannot be trusted (e.g., using these sensormeasurements will result in introducing errors into the estimatednavigational state).

Likewise, for the magnetometer, the expected model of sensor behaviorfor the magnetometer assumes that the local external magnetic field isuniform. If this assumption is violated (e.g., due to a local magneticdisturbance) the magnetometer measurements will be inaccurate andconsequently the navigational state estimate and the other processesthat depend on the navigational state estimate will be degraded. Morespecifically, the estimated navigational state will include erroneousgyroscope biases and/or erroneous headings angles (e.g., in the case ofpedestrian dead reckoning). Therefore, it is beneficial to detect anyuncompensated disturbances (e.g., a non-uniform disturbances in themagnetic field) and then to take steps to mitigate the effect of theresulting inaccuracies in magnetometer measurements on navigationalstate estimates for Device 102.

In this situation, the computer system switches to an alternative modeof operation (sometimes called “magnetic anomaly mode”), in which theeffect of the sensor measurements from the magnetometer on thenavigational state estimate is reduced. In the alternate mode ofoperation a gyroscope and/or one or more accelerometers are used toupdate navigational state estimates for Device 102 by integratingchanges in the acceleration or angular rotation to determine movement ofDevice 102. In some embodiments, while the navigational state estimatesfor Device 102 are being generated in the alternate mode of operation,sensor measurements from the magnetometer are ignored altogether untilthe measurement model becomes accurate again (e.g., the non-uniformdisturbance in the magnetic field is removed or ceases). In otherembodiments, while the estimates navigational state estimates for Device102 are being generated in the alternate mode of operation, the weightgiven to sensor measurements from the magnetometer is reduced until themeasurement model becomes accurate again (e.g., the non-uniformdisturbance in the magnetic field is removed).

FIG. 4 illustrates an example of switching between amagnetometer-assisted mode of operation and an alternate mode ofoperation. Device 102 starts in a first operating environment, OperatingEnvironment 1 (OE1) 402-1, which does not include a magnetic disturbancethat substantially degrades performance of the magnetometer. While inthe first operating environment, the magnetometer(s), accelerometer(s)and gyroscope(s) are used to update the navigational state of Device 102by a processing apparatus operating in the magnetometer-assisted mode ofoperation. Subsequently, Device 102 moves from the first operatingenvironment to a second operating environment, Operating Environment 2(OE2) 402-2, which does include a magnetic disturbance thatsubstantially degrades performance of the magnetometer (e.g., anon-uniform magnetic disturbance). For example Device 102 is placed on alarge metal table or near a speakerphone or other electronic device thatgenerates a strong magnetic field, the magnetic field near Device 102will be distorted and produce magnetic field measurements that differsubstantially from the reference magnetic field (e.g., the Earth'smagnetic field). While in the second operating environment, theaccelerometer(s) and gyroscope(s) are still used to update navigationalstate estimates for Device 102 by a processing apparatus operating inthe alternate mode of operation where the magnetometer(s) are not usedto updated the navigational state of Device 102. Subsequently, Device102 moves from the second operating environment to a third operatingenvironment, Operating Environment 3 (OE3) 402-3, which does not includea magnetic disturbance that substantially degrades performance of themagnetometer (e.g., a non-uniform magnetic disturbance). For exampleDevice 102 is lifted off of the large metal table or moved away from thespeakerphone or other electronic device that generates a strong magneticfield, so that the local magnetic field approximates the referencemagnetic field (e.g., the Earth's magnetic field). While in the firstoperating environment, the magnetometer(s), accelerometer(s) andgyroscope(s) are used to update navigational state estimates for Device102 by a processing apparatus operating in the magnetometer-assistedmode of operation (e.g., the processing apparatus returns to themagnetometer-assisted mode of operation).

In many situations when the non-uniform disturbance in the magneticfield is removed the measurement model will become accurate again (e.g.,because the navigational state estimate has not drifted substantiallyfrom when the sensor measurements from the magnetometer ceased to beused to update the estimate of the navigational state). In thesesituations, the processing apparatus can transition from the alternatemode of operation to the magnetometer-assisted mode of operation whenthe measurement model becomes accurate again (e.g., the magnetic fieldmeasured by the magnetometer is in agreement with the magnetic fieldpredicted based on measurements from the other sensors).

However, in other situations the estimate of the navigational statedrifts while in the alternate mode of operation (e.g., because thenavigation sensing device undergoes dynamic acceleration and/or thenon-uniform disturbance in the magnetic field is present for too long atime). In these situations, the accumulation of attitude drift caused byintegrating sensor measurements from the gyroscope and/oraccelerometer(s) will cause the measurement model to always report toohigh an error to ever recover (e.g., return to a magnetometer-assistedmode of operation) and thus it is difficult to determine whether or notit is appropriate to transition from the alternate mode of operation tothe magnetometer-assisted mode of operation. One possible approach torecovering from this situation where the magnetometer is not in use andthe estimate of the navigational state has drifted, is to determine ifthe external field is uniform, even if the measurement model is notaccurate, which indicates that the magnetometer is likely reliable butthat the navigational state estimate drifted while in the alternate modeof operation.

Determination of external field uniformity is possible by attempting toestimate the magnetic field in the inertial frame (“h_(ref)”) over avariety of diverse orientations. If the external magnetic field isuniform, then the estimate of the magnetic field in the inertial frame(“h_(ref)”) should have a variance comparable to the sensor noise of themagnetometer. Once the estimate of the magnetic field in the inertialframe (“h_(ref)”) has been generated, an attitude correction that willcompensate for the drift in the navigational state estimate can bedetermined and used to reintroduce the sensor measurements from themagnetometer to the navigational state estimation process, therebyreturning to the magnetometer-assisted mode of operation. An example ofone set of steps for transitioning from the alternate mode of operationto the magnetometer-assisted mode of operation is described below:

(1) Collect N “diverse” magnetometer measurements. A magnetometermeasurement m_(i) is diverse from other measurements if m_(i)^(T)<β∀j≠i, where β is a defined constant.

(2) Compute an estimate of the inertial frame magnetic field h_(ref)from magnetometer measurements and the filters associated attitudeestimates. (2a) Compute an inertial frame magnetic field direction foreach magnetometer measurement: h_(ref) ^((i))=C^(T)(q^(i))m_(i)/∥m_(i)∥where q_(i) is the attitude quaternion estimate. (2b) Compute anestimate of the inertial magnetic field direction through the mean:ĥ_(ref)=(1/N)Σ_(i) ^(N)h_(ref) ^((i)).

(3) Check the uniformity of the estimated magnetic field. In a uniformfield, the inertial frame magnetic field direction estimates fordifferent magnetometer measurements should be consistent. In someembodiments, checking the uniformity of the estimated field includeschecking the similarity of the individual estimates. The field isdefined as uniform if σ²<α, where σ²=(1/N)Σ_(i) ^(N)(1−ĥ_(ref)^(T)h_(ref) ^((i))) and α is defined as a constant.

(4) If Device 102 is determined to be in a uniform field, apply attitudecorrection and resume normal operating mode. (4a) In the situation wherethe reference magnetic field is the Earth's magnetic field, the inertialframe magnetic field direction estimated from this algorithm should becorrected to have zero azimuth. (4b) Compute rotation matrix totransform estimated magnetic field into zero azimuth:

$C = {\begin{bmatrix}{{\hat{h}}_{ref}(0)} & {{\hat{h}}_{ref}(1)} & 0 \\{- {{\hat{h}}_{ref}(1)}} & {{\hat{h}}_{ref}(0)} & 0 \\0 & 0 & 1\end{bmatrix}.}$(4c) Compute new magnetic field direction vector using above rotationmatrix, and feed the new magnetic field direction vector back to theattitude filter: h_(ref)=Cĥ_(ref). (4d) Compute an attitude correctionassociated with above azimuth correction to be applied to any previousattitude estimates dq=f(C). This fixed attitude correction is used tocorrect for the attitude drift that occurred while the magnetometer wasnot being used to update navigational state estimates for Device 102.

Attention is now directed to FIGS. 5A-5D, which illustrate a method 500for determining estimated navigational states of a navigation sensingdevice. Method 500 is, optionally, governed by instructions that arestored in a non-transitory computer readable storage medium and that areexecuted by one or more processors of one or more computer systems(e.g., Device 102, FIG. 6 or Host 101, FIG. 7). Each of the operationsshown in FIGS. 5A-5D typically corresponds to instructions stored in acomputer memory or non-transitory computer readable storage medium(e.g., Memory 1110 of Device 102 in FIG. 6 or Memory 1210 of Host 101 inFIG. 7). The computer readable storage medium optionally (and typically)includes a magnetic or optical disk storage device, solid state storagedevices such as Flash memory, or other non-volatile memory device ordevices. The computer readable instructions stored on the computerreadable storage medium typically include one or more of: source code,assembly language code, object code, or other instruction format that isinterpreted or executed by one or more processors. In variousembodiments, some operations in method 500 are combined and/or the orderof some operations is changed from the order shown in FIGS. 5A-5D.

The following operations are performed at a processing apparatus havingone or more processors and memory storing one or more programs that whenexecuted by the one or more processors cause the respective processingapparatus to perform the method. In some embodiments, the processingapparatus is a component of Device 102 (e.g., the processing apparatusincludes CPU(s) 1102 in FIG. 6). In some embodiments, the processingapparatus is separate from Device 102 (e.g., the processing apparatusincludes CPU(s) 1202 in FIG. 7).

The processing apparatus generates (502) navigational state estimatesfor a device having a plurality of sensors, the plurality of sensorsincluding a magnetometer. The processing apparatus has a plurality ofmodes of operation including: a magnetometer-assisted mode of operationin which measurements from the magnetometer are used to estimate thenavigational state of Device 102 and an alternate mode of operation inwhich measurements from a subset of sensors of the plurality of sensorsthat does not include the magnetometer are used to estimate thenavigational state of Device 102. In some embodiments, themagnetometer-assisted mode is also referred to as a first mode ofoperation, a primary mode of operation or a magnetometer-assistednavigation mode. In some embodiments, the alternate mode of operation isalso referred to as a second mode, a secondary mode, or adistorted/non-uniform magnetic field compensation navigation mode.

In some embodiments, a Kalman filter is used (504) to estimate thenavigational state of Device 102 in accordance with measurements fromsensors in the plurality of sensors and the Kalman filter includes amagnetic field residual term corresponding to a difference between theestimated magnetic field and the measured magnetic field. In someembodiments, while operating in the magnetometer-assisted mode ofoperation, the processing apparatus calculates (506) a first value ofthe magnetic field residual and use the first value of the magneticfield residual to estimate the navigational state of Device 102. Inother words, in the alternate mode of operation, the magnetometer isstill on and generating sensor measurements, but the sensor measurementsare not being used to update the navigational state and are, instead,being used to determine if and when the sensor measurements from themagnetometer can be reintroduced to the process of estimating thenavigational state of Device 102 (e.g., determining when to transitionfrom the alternate mode of operation to the magnetometer-assisted modeof operation).

In some embodiments, while operating in the alternate mode of operation,the processing apparatus calculates (508) a second value of the magneticfield residual and forgoes use of the second value of the magnetic fieldresidual to estimate the navigational state of Device 102. In someembodiments, the magnetic field residual term continues to be calculatedin the same manner without regard to whether or not the processingapparatus is operating in the magnetometer-assisted mode of operation orthe alternate mode of operation. However in the alternate mode ofoperation the magnetic field residual term is not used in updating theestimated navigational state of Device 102, while in themagnetometer-assisted mode of operation, the magnetic field residualterm is used to update the estimated navigational state of Device 102.An advantage of continuing to calculate the magnetic field residual termwhile operating in the alternate mode of operation is that the magneticfield residual term can be used to determine whether to switch back tothe magnetometer-assisted mode of operation. In other words, in somesituations the magnetic field residual term serves as a good proxy fordetermining whether a temporary magnetic disturbance has ended. In someembodiments, forgoing use of a value of the magnetic field residual(e.g., while operating in the alternate mode of operation) to estimatethe navigational state of Device 102 includes setting (510) a gain ofthe magnetic field residual to zero in the Kalman filter.

In some embodiments, the processing apparatus compares (512) theplurality of magnetometer measurements to one or more models ofpredefined magnetic disturbances and determines whether the plurality ofmagnetometer measurements are consistent with a respective model of theone or more models. In accordance with a determination that theplurality of magnetometer measurements are not (514) consistent with arespective model of the one or more models, the processing apparatuscontinues to operate in a current mode of operation and waits beforechecking again to determine whether the magnetometer measurements areconsistent with one of the models of magnetic disturbances. For example,if Device 102 is placed near a speakerphone on a conference table thatDevice 102 has not been placed near before, the processing apparatuswill not have any model for the magnetic disturbance caused by thatparticular speakerphone on that particular table and thus will not beable to adjust operation of the processing apparatus based on previouslystored information.

In contrast, in some embodiments, in accordance with a determinationthat the plurality of magnetometer measurements are (516) consistentwith a respective model of the one or more models. In this situation,the processing apparatus retrieves (518) information about a respectivepredefined magnetic disturbance that corresponds to the respectivemodel. After retrieving the information, the processing apparatusadjusts (520) operation of the processing apparatus in accordance withthe information about the respective predefined magnetic disturbance. Insome embodiments, adjusting operation of the processing apparatusincludes determining (522) a difference (e.g., angle, magnitude) betweenthe magnetic field measured by the magnetometer and a magnetic fieldpredicted in accordance with measurements from the subset of sensors andadjusting (524) the estimate of the navigational state of Device 102 inaccordance with the determined difference.

For example, if the processing apparatus determines that the pluralityof magnetometer measurements indicate that Device 102 is operating whileplugged in to a charger, or with a closed lid, different functionalityof Device 102 could be enabled or disabled accordingly (e.g., a displaycould be turned off if a lid of the device is closed and turned on ifthe lid of the device is opened). In these examples, the magneticdisturbances caused by a closed lid or being plugged into a powercharger can be modeled in advance and a predetermined compensation forthe known disturbance can be generated, so that it can be retrieved bythe processing apparatus when the known disturbance is detected. In someembodiments, the processing apparatus learns new magnetic disturbancesand is able to generate new models for detecting and compensating forrepeated magnetic disturbances, such as learning the characteristics ofa magnetic disturbance caused by Device 102 being placed in a particularvehicle (e.g., a vehicle owned by a user of Device 102).

In some implementations, for a prior time period that occurs prior tothe respective time period, the processing apparatus operates (526) inthe magnetometer-assisted mode of operation, using measurements from theplurality of sensors, including the magnetometer, to estimate thenavigational state of Device 102. During the prior time period, theprocessing apparatus determines whether a current operating environmentof Device 102 will distort measurements from the magnetometer (e.g., theprocessing apparatus determines whether there is a non-uniform magneticdisturbance that would reduce the accuracy of sensor measurements fromthe magnetometer below a predefined threshold). In accordance with adetermination that the current operating environment of Device 102 willnot (528) distort measurements from the magnetometer, the processingapparatus continues to operate in the magnetometer-assisted mode ofoperation (e.g., because the magnetometer-assisted mode of operationprovides navigational state estimates that do not accumulate drift overtime, in contrast to navigational state estimates based on gyroscopeand/or accelerometer measurements). In contrast, in accordance with adetermination that the current operating environment of Device 102 will(530) distort measurements from the magnetometer, the processingapparatus transitions (532) to the alternate mode of operation in whichmeasurements from the magnetometer are not used to estimate thenavigational state of Device 102 (e.g., while Device 102 is in thecurrent operating environment that includes the magnetic disturbance).

For a respective time period, the processing apparatus operates in thealternate mode of operation. During the respective time period, theprocessing apparatus collects (534) a plurality of magnetometermeasurements and determines (536) whether the plurality of magnetometermeasurements meet predefined measurement-consistency requirements. Inother words, while the processing apparatus is in the alternate mode ofoperation, the processing apparatus collects sensor measurements thatare to be used to determine whether or not it is safe to resume usingsensor measurements from the magnetometer to estimate the navigationalstate of Device 102.

In some implementations, determining whether the plurality ofmagnetometer measurements meet predefined measurement-consistencyrequirements includes determining whether the magnetometer measurementsare consistent with other sensor measurements (e.g., comparativeconsistency of the magnetometer measurements) and, if the magnetometermeasurements are not consistent with other sensor measurements,determining whether the magnetometer measurements are consistent withother magnetometer measurements (e.g., internal consistency of themagnetometer measurements). However, in some embodiments the processingapparatus does not check for comparative consistency of the magnetometermeasurements and/or does not check for internal consistency of themagnetometer measurements.

In some implementations the processing apparatus determines whether theplurality of magnetometer measurements correspond to a first estimatedmagnetic field that is consistent with a second estimated magnetic fielddetermined based on measurements from one or more other sensors. Inparticular, while in the alternate mode of operation, the processingapparatus determines a first estimated direction of the externalmagnetic field based on the plurality of magnetometer measurements(e.g., an estimate of the local magnetic field based on a diverse set ofmagnetometer measurements). Additionally, while in the alternate mode ofoperation, the processing apparatus also determines a second estimateddirection of the external magnetic field based on measurements from oneor more of the other sensors (e.g., an integration of accelerometerand/or gyroscope measurements that indicate relative movement of Device102 from a respective navigational state) and a predeterminedrelationship between an estimated orientation of Device 102 and theexternal magnetic field (e.g., a previously determined relationshipbetween the respective navigational state and the reference magneticfield). After determining the first estimated direction and the secondestimated direction, the processing apparatus compares the firstestimated direction with the second estimated direction. As an example,the predetermined relationship is a relationship between the respectivenavigational state and the reference magnetic field determined prior todetecting the magnetic disturbance and transitioning to the alternatemode of operation. In this example, the measurements from the one ormore sensors are measurements from an accelerometer and/or gyroscopethat are integrated to determine changes to the navigational state ofDevice 102 since the predefined relationship was determined. Bycombining a relationship between the magnetic field and an initialnavigational state with information indicating a difference between theinitial navigational state and the current navigational state, a currentrelationship between the magnetic field and the current navigationalstate can be estimated (e.g., by assuming that no drift in thenavigational state has occurred).

In some embodiments, in accordance with the determination that the firstestimated magnetic field is (538) consistent with the second estimatedmagnetic field, the processing apparatus determines that the pluralityof magnetometer measurements meet (540) the predefined measurementconsistency requirements. In other words, the processing apparatusdetermines that there was a temporary disturbance in the externalmagnetic field, which has ended and whether the processing apparatus cansafely (e.g., without reducing the accuracy of the navigational stateestimate for Device 102) transition back to the magnetometer-assistedmode of operation, as described in greater detail below with referenceto operations 554-562.

In contrast, in accordance with the determination that the firstestimated magnetic field is not (542) consistent with the secondestimated magnetic field, the processing apparatus determines whetherthe plurality of magnetometer measurements are internally consistent. Inother words, the processing apparatus determines that the magnetic fieldis substantially uniform but that, due to the fact that the estimatednavigational state of Device 102 has drifted, the estimate of themagnetic field generated based on the navigational state of Device 102does not match the estimate of the magnetic field generated based on themagnetometer (e.g., because the magnetic disturbance had a longduration, which allowed an accelerometer or gyroscope determinednavigational state to drift). In this situation it is unlikely that acomparative consistency determination will ever return a match (e.g.,because it is statistically unlikely that a navigational state estimateof Device 102 based on the accelerometer and/or gyroscope measurementswill drift back into alignment with the magnetic field). Thus, in someembodiments an internal consistency comparison is performed to determinewhether the magnetometer measurements are consistent with each other.

In some embodiments, the internal consistency comparison determineswhether the direction of the magnetometer measurements is consistent. Inparticular, the processing apparatus determines a set of estimateddirections of an external magnetic field based on correspondingmagnetometer measurements and the plurality of magnetometer measurementsare determined to meet the predefined measurement-consistencyrequirements when the set of estimated directions have a statisticaldispersion below a predefined dispersion threshold (e.g., a standarddeviation below a predefined numerical threshold). In some embodiments,the plurality of magnetometer measurements are collected so as to have astatistical dispersion below the predefined dispersion threshold. Forexample, when Device 102 is in a particular orientation, a magnetometermeasurement is not collected at the particular orientation of Device 102if another magnetometer measurement has already been collected at asubstantially similar orientation (e.g., an orientation within 10 or 15degrees of the current orientation). Alternatively, in someimplementations a large number of magnetometer measurements arecollected and only a subset of the collected magnetometer measurementsare used. For example, the subset of the collected magnetometermeasurements can be selected so as to exclude one or more magnetometermeasurements that do not add additional measurement diversity.

In some embodiments, the internal consistency comparison determineswhether the magnitude of the magnetometer measurements is consistent. Inparticular, the processing apparatus determines a set of estimatedmagnitudes of an external magnetic field based on correspondingmagnetometer measurements and the plurality of magnetometer measurementsmeet the predefined measurement-consistency requirements when the set ofestimated magnitudes have a statistical dispersion below a predefineddispersion threshold. (e.g., a standard deviation below a predefinednumerical threshold). In some embodiments, the plurality of magnetometermeasurements are collected so as to have a statistical dispersion belowthe predefined dispersion threshold. For example, when Device 102 is ina particular orientation, a magnetometer measurement is not collected atthe particular orientation of Device 102 if another magnetometermeasurement has already been collected at a substantially similarorientation (e.g., an orientation within 10 or 15 degrees of the currentorientation). Alternatively, in some implementations a large number ofmagnetometer measurements are collected and only a subset of thecollected magnetometer measurements are used. For example, the subset ofthe collected magnetometer measurements can be selected so as to excludeone or more magnetometer measurements that do not add additionalmeasurement diversity.

In accordance with a determination that the plurality of magnetometermeasurements are (544) internally consistent, the processing apparatusdetermines that the plurality of magnetometer measurements meet (540)the predefined measurement consistency requirements and that theprocessing apparatus can safely (e.g., without reducing the accuracy ofthe navigational state estimate for Device 102) transition back to themagnetometer-assisted mode of operation, as described in greater detailbelow with reference to operations 554-562.

In accordance with a determination that the plurality of magnetometermeasurements are not (546) internally consistent, the processingapparatus determines that the plurality of magnetometer measurements donot meet (548) the predefined measurement consistency requirements. Inaccordance with the determination that the plurality of magnetometermeasurements do not meet (550) predefined measurement-consistencyrequirements, the processing apparatus continues to operate in thealternate mode of operation. Optionally, continuing to operate in thealternate mode of operation includes repeatedly determining whether itis safe to transition from the alternate mode of operation to themagnetometer-assisted mode of operation (e.g., at periodic intervals orin response to detecting a triggering condition such as when a localmagnetic field in the proximity of the device changes).

In contrast, in accordance with a determination that the plurality ofmagnetometer measurements meet (552) predefined measurement-consistencyrequirements, the processing apparatus transitions (554) to themagnetometer-assisted mode of operation. In some embodiments,transitioning from the alternate mode of operation to themagnetometer-assisted mode of operation includes determining (556) adifference (e.g., angle, magnitude) between the magnetic field measuredby the magnetometer and a magnetic field predicted in accordance withmeasurements from the subset of sensors. In these embodiments, afterdetermining the difference between the measured magnetic field and thepredicted magnetic field, the processing apparatus adjusts (558) theestimate of the navigational state of Device 102 in accordance with thedetermined difference (e.g., by computing a rotational matrix totransform the estimated magnetic field into zero azimuth).

In some implementations, after transitioning to themagnetometer-assisted mode of operation, the processing apparatus uses(560) measurements of the plurality of sensors, including themagnetometer, to estimate the navigational state of Device 102. In someembodiments, while the processing apparatus is in the alternate mode ofoperation, a gain of the magnetic field residual is set to zero in theKalman filter and in accordance with a determination that the magneticfield residual has decreased below a predefined minimum threshold theprocessing apparatus increases (562) the gain of the magnetic fieldresidual to a non-zero value. In other words, in some embodiments, themagnetic field residual is computed in both the magnetometer-assistedmode of operation and the alternate mode of operation but is used fordifferent purposes in the two different modes of operation. For example,while in the magnetometer-assisted mode of operation the processingapparatus uses the magnetic field residual to update navigational stateestimates for Device 102. In contrast, while in the alternate mode ofoperation the processing apparatus uses the magnetic field residual todetermine whether or not to transition back to the magnetometer-assistedmode of operation. Moreover, removing the magnetic field residual andreintroducing the magnetic field residual from the navigational stateestimation process (e.g., the Kalman filter) can be accomplished byadjusting a gain of the magnetic field residual term in the Kalmanfilter rather than by transitioning between two different Kalmanfilters.

It should be understood that the particular order in which theoperations in FIGS. 5A-5D have been described are merely exemplary andare not intended to indicate that the described order is the only orderin which the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein.

System Structure

FIG. 6 is a block diagram of Navigation sensing Device 102 (herein“Device 102”). Device 102 typically includes one or more processingunits (CPUs) 1102, one or more network or other CommunicationsInterfaces 1104 (e.g., a wireless communication interface, as describedabove with reference to FIG. 1), Memory 1110, Sensors 1168 (e.g.,Sensors 220 such as one or more Accelerometers 1170, Magnetometers 1172,Gyroscopes 1174, Beacon Sensors 1176, Inertial Measurement Units 1178,Thermometers, Barometers, and/or Proximity Sensors, etc.), one or moreCameras 1180, and one or more Communication Buses 1109 forinterconnecting these components. In some embodiments, CommunicationsInterfaces 1104 include a transmitter for transmitting information, suchas accelerometer and magnetometer measurements, and/or the computednavigational state of Device 102, and/or other information to Host 101.Communication buses 1109 typically include circuitry (sometimes called achipset) that interconnects and controls communications between systemcomponents. Device 102 optionally includes user interface 1105comprising Display 1106 (e.g., Display 104 in FIG. 1) and Input Devices1107 (e.g., keypads, buttons, etc.). Memory 1110 includes high-speedrandom access memory, such as DRAM, SRAM, DDR RAM or other random accesssolid state memory devices; and may include non-volatile memory, such asone or more magnetic disk storage devices, optical disk storage devices,flash memory devices, or other non-volatile solid state storage devices.Memory 1110 optionally includes one or more storage devices remotelylocated from the CPU(s) 1102. Memory 1110, or alternately thenon-volatile memory device(s) within Memory 1110, comprises anon-transitory computer readable storage medium. In some embodiments,Memory 1110 stores the following programs, modules and data structures,or a subset thereof:

-   -   Operating System 1112 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   Communication Module 1113 that is used for connecting Device 102        to Host 101 via Communication Network Interface(s) 1104 (wired        or wireless); Communication Module 1113 is optionally adapted        for connecting Device 102 to one or more communication networks,        such as the Internet, other wide area networks, local area        networks, metropolitan area networks, and so on;    -   Sensor Measurements 1114 (e.g., data representing accelerometer        measurements, magnetometer measurements, gyroscope measurements,        global positioning system measurements, beacon sensor        measurements, inertial measurement unit measurements,        thermometer measurements, atmospheric pressure measurements,        proximity measurements, etc.);    -   data representing Button Presses 1116;    -   Magnetic Disturbance Detector 1130 for detecting disturbances in        the local magnetic field of Device 102 (e.g., detecting sudden        changes in magnetic field direction that do not correspond to        changes in navigational state of Device 102 and/or detecting        that the local magnetic field is non-uniform);    -   Mode of Operation Selector 1132, for selecting a mode of        operation for the processing apparatus (e.g., the        magnetometer-assisted mode or the alternate mode), which        optionally includes Comparative Consistency Module 1134 for        determining whether magnetometer measurements are consistent        with other sensor measurements and Internal Consistency Module        1136 for determining whether magnetometer measurements are        internally consistent (e.g., that Device 102 is in a uniform        magnetic field);    -   Navigational State Compensator 1138 for determining a fixed        compensation (e.g., a rotational offset) for compensating for        drift in the navigational state estimate while the processing        apparatus was in the alternate mode of operation;    -   Navigation State Estimator 1140 for estimating navigational        states of Device 102, optionally including:        -   Kalman Filter Module 1142 that determines the attitude of            Device 102, as described in U.S. Pat. Pub. No. 2010/0174506            Equations 8-29, wherein the Kalman filter module includes: a            sensor model (e.g., the sensor model described in Equations            28-29 of U.S. Pat. Pub. No. 2010/0174506), a dynamics model            (e.g., the dynamics model described in Equations 15-21 of            U.S. Pat. Pub. No. 2010/0174506), a predict module that            performs the predict phase operations of the Kalman filter,            an update module that performs the update operations of the            Kalman filter, a state vector of the Kalman filter (e.g.,            the state vector {circumflex over (x)} in Equation 10 of            U.S. Pat. Pub. No. 2010/0174506), a mapping, Kalman filter            matrices, and attitude estimates (e.g., the attitude            estimates as obtained from the quaternion in the state            vector {circumflex over (x)} in Equation 10 of U.S. Pat.            Pub. No. 2010/0174506);        -   Magnetic Field Residual 1144 that is indicative of a            difference between a magnetic field detected based on            measurements from Magnetometer(s) 1172 and a magnetic field            estimated based on Kalman Filter Module 1142; and        -   data representing Navigational State Estimate 1150 (e.g., an            estimate of the position and/or attitude of Device 102).    -   optionally, User Interface Module 1152 that receives commands        from the user via Input Device(s) 1107 and generates user        interface objects in Display(s) 1106 in accordance with the        commands and the navigational state of Device 102, User        Interface Module 1152 optionally includes one or more of: a        cursor position module for determining a cursor position for a        cursor to be displayed in a user interface in accordance with        changes in a navigational state of the navigation sensing        device, an augmented reality module for determining positions of        one or more user interface objects to be displayed overlaying a        dynamic background such as a camera output in accordance with        changes in a navigational state of the navigation sensing        device, a virtual world module for determining a portion of a        larger user interface (a portion of a virtual world) to be        displayed in accordance with changes in a navigational state of        the navigation sensing device, a pedestrian dead reckoning        module for tracking movement of Device 102 over time, and other        application specific user interface modules; and    -   optionally, Gesture Determination Module 1154 for determining        gestures in accordance with detected changes in the navigational        state of Device 102.

It is noted that in some of the embodiments described above, Device 102does not include a Gesture Determination Module 1154, because gesturedetermination is performed by Host 101. In some embodiments describedabove, Device 102 also does not include Magnetic Disturbance Detector1130, Mode of Operation Selector 1132, Navigational State Estimator 1140and User Interface Module because Device 102 transmits SensorMeasurements 1114 and, optionally, data representing Button Presses 1116to a Host 101 at which a navigational state of Device 102 is determined.

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices, and each of the aboveidentified programs or modules corresponds to a set of instructions forperforming a function described above. The set of instructions can beexecuted by one or more processors (e.g., CPUs 1102). The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures or modules, andthus various subsets of these modules may be combined or otherwisere-arranged in various embodiments. In some embodiments, Memory 1110 maystore a subset of the modules and data structures identified above.Furthermore, Memory 1110 may store additional modules and datastructures not described above.

Although FIG. 6 shows a “Navigation sensing Device 102,” FIG. 6 isintended more as functional description of the various features whichmay be present in a navigation sensing device. In practice, and asrecognized by those of ordinary skill in the art, items shown separatelycould be combined and some items could be separated.

FIG. 7 is a block diagram of Host Computer System 101 (herein “Host101”). Host 101 typically includes one or more processing units (CPUs)1202, one or more network or other Communications Interfaces 1204 (e.g.,any of the wireless interfaces described above with reference to FIG.1), Memory 1210, and one or more Communication Buses 1209 forinterconnecting these components. In some embodiments, CommunicationInterfaces 1204 include a receiver for receiving information, such asaccelerometer and magnetometer measurements, and/or the computedattitude of a navigation sensing device (e.g., Device 102), and/or otherinformation from Device 102. Communication Buses 1209 optionally includecircuitry (sometimes called a chipset) that interconnects and controlscommunications between system components. Host 101 optionally includes aUser Interface 1205 comprising a Display 1206 (e.g., Display 104 inFIG. 1) and Input Devices 1207 (e.g., a navigation sensing device suchas a multi-dimensional pointer, a mouse, a keyboard, a trackpad, atrackball, a keypad, buttons, etc.). Memory 1210 includes high-speedrandom access memory, such as DRAM, SRAM, DDR RAM or other random accesssolid state memory devices; and may include non-volatile memory, such asone or more magnetic disk storage devices, optical disk storage devices,flash memory devices, or other non-volatile solid state storage devices.Memory 1210 optionally includes one or more storage devices remotelylocated from the CPU(s) 1202. Memory 1210, or alternately thenon-volatile memory device(s) within Memory 1210, comprises anon-transitory computer readable storage medium. In some embodiments,Memory 1210 stores the following programs, modules and data structures,or a subset thereof:

-   -   Operating System 1212 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   Communication Module 1213 that is used for connecting Host 101        to Device 102, and/or other devices or systems via Communication        Network Interface(s) 1204 (wired or wireless), and for        connecting Host 101 to one or more communication networks, such        as the Internet, other wide area networks, local area networks,        metropolitan area networks, and so on;    -   Sensor Measurements 1214 (e.g., data representing accelerometer        measurements, magnetometer measurements, gyroscope measurements,        global positioning system measurements, beacon sensor        measurements, inertial measurement unit measurements,        thermometer measurements, atmospheric pressure measurements,        proximity measurements, etc.);    -   data representing Button Presses 1216;    -   Magnetic Disturbance Detector 1230 for detecting disturbances in        the local magnetic field of Device 102 (e.g., detecting sudden        changes in magnetic field direction that do not correspond to        changes in navigational state of Device 102 and/or detecting        that the local magnetic field is non-uniform);    -   Mode of Operation Selector 1232, for selecting a mode of        operation for the processing apparatus (e.g., the        magnetometer-assisted mode or the alternate mode), which        optionally includes Comparative Consistency Module 1234 for        determining whether magnetometer measurements for Device 102 are        consistent with other sensor measurements for Device 102 and        Internal Consistency Module 1236 for determining whether        magnetometer measurements are internally consistent (e.g., that        Device 102 is in a uniform magnetic field);    -   Navigational State Compensator 1238 for determining a fixed        compensation (e.g., a rotational offset) for compensating for        drift in the navigational state estimate of Device 102 while the        processing apparatus was in the alternate mode of operation;    -   Navigation State Estimator 1240 for estimating navigational        states of Device 102, optionally including:        -   Kalman Filter Module 1242 that determines the attitude of            Device 102, as described in U.S. Pat. Pub. No. 2010/0174506            Equations 8-29, wherein the Kalman filter module includes: a            sensor model (e.g., the sensor model described in Equations            28-29 of U.S. Pat. Pub. No. 2010/0174506), a dynamics model            (e.g., the dynamics model described in Equations 15-21 of            U.S. Pat. Pub. No. 2010/0174506), a predict module that            performs the predict phase operations of the Kalman filter,            an update module that performs the update operations of the            Kalman filter, a state vector of the Kalman filter (e.g.,            the state vector {circumflex over (x)} in Equation 10 of            U.S. Pat. Pub. No. 2010/0174506), a mapping, Kalman filter            matrices, and attitude estimates (e.g., the attitude            estimates as obtained from the quaternion in the state            vector {circumflex over (x)} in Equation 10 of U.S. Pat.            Pub. No. 2010/0174506);        -   Magnetic Field Residual 1244 that is indicative of a            difference between a magnetic field detected based on            measurements from Magnetometer(s) 1272 and a magnetic field            estimated based on Kalman Filter Module 1242; and        -   data representing Navigational State Estimate 1250 (e.g., an            estimate of the position and/or attitude of Device 102).    -   optionally, User Interface Module 1252 that receives commands        from the user via Input Device(s) 1207 and generates user        interface objects in Display(s) 1206 in accordance with the        commands and the navigational state of Device 102, User        Interface Module 1252 optionally includes one or more of: a        cursor position module for determining a cursor position for a        cursor to be displayed in a user interface in accordance with        changes in a navigational state of the navigation sensing        device, an augmented reality module for determining positions of        one or more user interface objects to be displayed overlaying a        dynamic background such as a camera output in accordance with        changes in a navigational state of the navigation sensing        device, a virtual world module for determining a portion of a        larger user interface (a portion of a virtual world) to be        displayed in accordance with changes in a navigational state of        the navigation sensing device, a pedestrian dead reckoning        module for tracking movement of Device 102 over time, and other        application specific user interface modules; and    -   optionally, Gesture Determination Module 1254 for determining        gestures in accordance with detected changes in the navigational        state of Device 102.

It is noted that in some of the embodiments described above, Host 101does not store data representing Sensor Measurements 1214, becausesensor measurements of Device 102 are processed at Device 102, whichsends data representing Navigational State Estimate 1250 to Host 101. Inother embodiments, Device 102 sends data representing SensorMeasurements 1214 to Host 101, in which case the modules for processingthat data are present in Host 101.

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices, and each of the aboveidentified programs or modules corresponds to a set of instructions forperforming a function described above. The set of instructions can beexecuted by one or more processors (e.g., CPUs 1202). The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures or modules, andthus various subsets of these modules may be combined or otherwisere-arranged in various embodiments. The actual number of processors andsoftware modules used to implement Host 101 and how features areallocated among them will vary from one implementation to another. Insome embodiments, Memory 1210 may store a subset of the modules and datastructures identified above. Furthermore, Memory 1210 may storeadditional modules and data structures not described above.

Note that method 500 described above is optionally governed byinstructions that are stored in a non-transitory computer readablestorage medium and that are executed by one or more processors of Device102 or Host 101. As noted above, in some embodiments these methods maybe performed in part on Device 102 and in part on Host 101, or on asingle integrated system which performs all the necessary operations.Each of the operations shown in FIGS. 5A-5D optionally correspond toinstructions stored in a computer memory or computer readable storagemedium. The computer readable storage medium optionally includes amagnetic or optical disk storage device, solid state storage devicessuch as Flash memory, or other non-volatile memory device or devices. Insome embodiments, the computer readable instructions stored on thecomputer readable storage medium are in source code, assembly languagecode, object code, or other instruction format that is interpreted orexecuted by one or more processors.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best utilize the invention andvarious embodiments with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A method comprising: at a processing apparatushaving one or more processors and memory storing one or more programsthat when executed by the one or more processors cause the respectiveprocessing apparatus to perform the method, generating navigationalstate estimates for a device having a plurality of sensors, theplurality of sensors including a magnetometer, wherein the processingapparatus has a plurality of modes of operation including: amagnetometer-assisted mode of operation in which measurements from themagnetometer are used to estimate the navigational state of the device;and an alternate mode of operation in which measurements from a subsetof sensors of the plurality of sensors that does not include themagnetometer are used to estimate the navigational state of the device;for a respective time period, operating in the alternate mode ofoperation; during the respective time period: collecting a plurality ofmagnetometer measurements; determining a first estimated direction ofthe external magnetic field based on the plurality of magnetometermeasurements; determining a second estimated direction of the externalmagnetic field based on measurements from one or more of the othersensors and a predetermined relationship between an estimatedorientation of the device and the external magnetic field; anddetermining whether the plurality of magnetometer measurements meetpredefined measurement-consistency requirements while the firstestimated magnetic field is not consistent with the second estimatedmagnetic field; in accordance with the determination that the pluralityof magnetometer measurements meet predefined measurement-consistencyrequirements while the first estimated magnetic field is not consistentwith the second estimated magnetic field, transitioning to themagnetometer-assisted mode of operation, including adjusting theestimate of the navigational state of the device in accordance with adifference between the first estimated magnetic field and the secondestimated magnetic field; and in accordance with the determination thatthe plurality of magnetometer measurements do not meet predefinedmeasurement-consistency requirements, continuing to operate in thealternate mode of operation.
 2. The method of claim 1, wherein theprocessing apparatus is a component of the device.
 3. The method ofclaim 1, further comprising: for a prior time period that occurs priorto the respective time period, operating in the magnetometer-assistedmode of operation, using measurements from the plurality of sensors,including the magnetometer, to estimate the navigational state of thedevice; during the prior time period, determining whether a currentoperating environment of the device will distort measurements from themagnetometer; and in accordance with a determination that the currentoperating environment of the device will distort measurements from themagnetometer, transitioning to the alternate mode of operation in whichmeasurements from the magnetometer are not used to estimate thenavigational state of the device.
 4. The method of claim 1, furthercomprising, after transitioning to the magnetometer-assisted mode ofoperation, using measurements of the plurality of sensors, including themagnetometer, to estimate the navigational state of the device.
 5. Themethod of claim 1, wherein: a set of estimated directions of theexternal magnetic field are determined based on correspondingmagnetometer measurements; and the plurality of magnetometermeasurements meet the predefined measurement-consistency requirementswhen the set of estimated directions have a statistical dispersion belowa predefined dispersion threshold.
 6. The method of claim 1, wherein: aset of estimated magnitudes of the external magnetic field aredetermined based on corresponding magnetometer measurements; and theplurality of magnetometer measurements meet the predefinedmeasurement-consistency requirements when the set of estimatedmagnitudes have a statistical dispersion below a predefined dispersionthreshold.
 7. The method of claim 1, further comprising, while in thealternate mode of operation comparing the first estimated direction withthe second estimated direction, wherein the plurality of magnetometermeasurements meet the predefined measurement-consistency requirementswhen the first estimated direction is consistent with the secondestimated direction.
 8. The method of claim 1, wherein determiningwhether the plurality of magnetometer measurements meet predefinedmeasurement-consistency requirements includes determining whether theplurality of magnetometer measurements are internally consistent.
 9. Themethod of claim 1, wherein: a Kalman filter is used to estimate thenavigational state of the device in accordance with measurements fromsensors in the plurality of sensors; the Kalman filter includes amagnetic field residual term corresponding to a difference between theestimated magnetic field and the measured magnetic field; and the methodfurther comprises: while operating in the magnetometer-assisted mode ofoperation: calculating a first value of the magnetic field residual; andusing the first value of the magnetic field residual to estimate thenavigational state of the device; and while operating in the alternatemode of operation: calculating a second value of the magnetic fieldresidual; and forgoing use of the second value of the magnetic fieldresidual to estimate the navigational state of the device.
 10. Themethod of claim 9, wherein: forgoing use of a value of the magneticfield residual to estimate the navigational state of the device includessetting a gain of the magnetic field residual to zero in the Kalmanfilter; and the method further comprises, in accordance with adetermination that the magnetic field residual has decreased below apredefined minimum threshold, increasing the gain of the magnetic fieldresidual to a non-zero value.
 11. The method of claim 1, furthercomprising: comparing the plurality of magnetometer measurements to oneor more models of predefined magnetic disturbances; and in accordancewith a determination that the plurality of magnetometer measurements areconsistent with a respective model of the one or more models: retrievinginformation about a respective predefined magnetic disturbance thatcorresponds to the respective model; and adjusting operation of theprocessing apparatus in accordance with the information about therespective predefined magnetic disturbance.
 12. The method of claim 11,wherein adjusting operation of the processing apparatus includes:determining a difference between the magnetic field measured by themagnetometer and a magnetic field predicted in accordance withmeasurements from the subset of sensors; and adjusting the estimate ofthe navigational state of the device in accordance with the determineddifference.
 13. A computer system, comprising: a processing apparatusincluding one or more processors and memory; and one or more programs,wherein the one or more programs are stored in the memory and configuredto be executed by the one or more processors, the one or more programsincluding instructions for: generating navigational state estimates fora device having a plurality of sensors, the plurality of sensorsincluding a magnetometer, wherein the processing apparatus has aplurality of modes of operation including: a magnetometer-assisted modeof operation in which measurements from the magnetometer are used toestimate the navigational state of the device; and an alternate mode ofoperation in which measurements from a subset of sensors of theplurality of sensors that does not include the magnetometer are used toestimate the navigational state of the device; and for a respective timeperiod, operating the processing apparatus in the alternate mode ofoperation; during the respective time period: collecting a plurality ofmagnetometer measurements; determining a first estimated direction ofthe external magnetic field based on the plurality of magnetometermeasurements; determining a second estimated direction of the externalmagnetic field based on measurements from one or more of the othersensors and a predetermined relationship between an estimatedorientation of the device and the external magnetic field; anddetermining whether the plurality of magnetometer measurements meetpredefined measurement-consistency requirements while the firstestimated magnetic field is not consistent with the second estimatedmagnetic field; in accordance with the determination that the pluralityof magnetometer measurements meet predefined measurement-consistencyrequirements while the first estimated magnetic field is not consistentwith the second estimated magnetic field, transitioning to themagnetometer-assisted mode of operation, including adjusting theestimate of the navigational state of the device in accordance with adifference between the first estimated magnetic field and the secondestimated magnetic field; and in accordance with the determination thatthe plurality of magnetometer measurements do not meet predefinedmeasurement-consistency requirements, continuing to operate theprocessing apparatus in the alternate mode of operation.
 14. Thecomputer system of claim 13, wherein the one or more programs includeinstructions for: for a prior time period that occurs prior to therespective time period, operating in the magnetometer-assisted mode ofoperation, using measurements from the plurality of sensors, includingthe magnetometer, to estimate the navigational state of the device;during the prior time period, determining whether a current operatingenvironment of the device will distort measurements from themagnetometer; and in accordance with a determination that the currentoperating environment of the device will distort measurements from themagnetometer, transitioning to the alternate mode of operation in whichmeasurements from the magnetometer are not used to estimate thenavigational state of the device.
 15. The computer system of claim 13,wherein the one or more programs include instructions for, while in thealternate mode of operation comparing the first estimated direction withthe second estimated direction, wherein the plurality of magnetometermeasurements meet the predefined measurement-consistency requirementswhen the first estimated direction is consistent with the secondestimated direction.
 16. The computer system of claim 13, whereindetermining whether the plurality of magnetometer measurements meetpredefined measurement-consistency requirements includes determiningwhether the plurality of magnetometer measurements are internallyconsistent.
 17. A non-transitory computer readable storage mediumstoring one or more programs, the one or more programs comprisinginstructions, which when executed by a processing apparatus with one ormore processors, cause the processing apparatus to: generatenavigational state estimates for a device having a plurality of sensors,the plurality of sensors including a magnetometer, wherein theprocessing apparatus has a plurality of modes of operation including: amagnetometer-assisted mode of operation in which measurements from themagnetometer are used to estimate the navigational state of the device;and an alternate mode of operation in which measurements from a subsetof sensors of the plurality of sensors that does not include themagnetometer are used to estimate the navigational state of the device;and for a respective time period, operate the processing apparatus inthe alternate mode of operation; during the respective time period:collect a plurality of magnetometer measurements; determine a firstestimated direction of the external magnetic field based on theplurality of magnetometer measurements; determine a second estimateddirection of the external magnetic field based on measurements from oneor more of the other sensors and a predetermined relationship between anestimated orientation of the device and the external magnetic field; anddetermine whether the plurality of magnetometer measurements meetpredefined measurement-consistency requirements while the firstestimated magnetic field is not consistent with the second estimatedmagnetic field; in accordance with the determination that the pluralityof magnetometer measurements meet predefined measurement-consistencyrequirements while the first estimated magnetic field is not consistentwith the second estimated magnetic field, transition to themagnetometer-assisted mode of operation, including adjusting theestimate of the navigational state of the device in accordance with adifference between the first estimated magnetic field and the secondestimated magnetic field; and in accordance with the determination thatthe plurality of magnetometer measurements do not meet predefinedmeasurement-consistency requirements, continue to operate the processingapparatus in the alternate mode of operation.
 18. The non-transitorycomputer readable storage medium of claim 17, further comprisinginstructions to: for a prior time period that occurs prior to therespective time period, operating in the magnetometer-assisted mode ofoperation, use measurements from the plurality of sensors, including themagnetometer, to estimate the navigational state of the device; duringthe prior time period, determine whether a current operating environmentof the device will distort measurements from the magnetometer; and inaccordance with a determination that the current operating environmentof the device will distort measurements from the magnetometer,transition to the alternate mode of operation in which measurements fromthe magnetometer are not used to estimate the navigational state of thedevice.
 19. The non-transitory computer readable storage medium of claim17, further comprising instructions to compare the first estimateddirection with the second estimated direction, wherein the plurality ofmagnetometer measurements meet the predefined measurement-consistencyrequirements when the first estimated direction is consistent with thesecond estimated direction.
 20. The non-transitory computer readablestorage medium of claim 17, wherein determining whether the plurality ofmagnetometer measurements meet predefined measurement-consistencyrequirements includes determining whether the plurality of magnetometermeasurements are internally consistent.